Validation Guidelines for IS Positivist Research

Fourteen years ago, Straub (1989) raised the issue of whether IS positivist researchers were sufficiently validating their instruments. Since then, the IS profession has been exposed to many opportunities and challenges. E-Commerce has risen in prominence. Other management trends have passed through their cycles. Our new professional society, the Association of Information Systems, has been formed and amalgamated with the preeminent research conference in the field, the International Conference on Information Systems. But the issue of rigor in IS research is still one of our most critical scientific issues. Without solid validation of the instruments that are used to gather data upon which findings and interpretations are based, the very scientific basis of the profession is threatened. This study builds on two prior retrospectives of IS research that conclude that IS positivist researchers still have major barriers to overcome in instrument, statistical, and other forms of validation. It goes beyond these studies by offering analyses of the state-of-the-art of research validities and deriving specific heuristics for research practice in the validities. Some of these heuristics will, no doubt, be controversial. But we believe that it is time for the IS academic profession to bring such issues into the open for community debate, and this article is an initial step in that direction. Based on our interpretation of the importance of a long list of validities, this paper suggests heuristics for reinvigorating the quest for validation in IS research via content/construct validity, reliability, manipulation validity, and statistical conclusion validity. New guidelines for validation and new research directions conclude the paper.